Diversity/Parallelism Trade-Off in Distributed Systems With Redundancy

Distributed computing enables parallel execution of smaller tasks that make up a large job. Its purpose is to reduce the job completion time. However, random fluctuations in task service times lead straggling with long times. Redundancy provides diversity allows when only subset redundant executed, thus removing dependency on tasks. Under constrained resources (here, fixed number parallel servers), increasing redundancy reduces available for parallelism. In this paper, we characterize diversity vs. parallelism trade-off and identify optimal strategy among replication, coding, splitting, which minimizes expected We consider three common time distributions establish models describe scaling these size. find different operate optimally at levels, requiring very code rates.